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REVIEW AND EVALUATION OF REQUESTS 



The U.S. Department of Education (Department) will use a review process that will include both external peer reviewers and staff reviewers to evaluate 
State educational agency (SEA) requests for this flexibility. This review process will help ensure that each request for this flexibility approved by the 
Department is consistent with the principles, which are designed to support State efforts to improve student academic achievement and increase the 
quality of instruction, and is both educationally and technically sound. Reviewers will evaluate whether and how each request for this flexibility will 
support a comprehensive and coherent set of improvements in the areas of standards and assessments, accountability, and teacher and principal 
effectiveness that will lead to improved student outcomes. Each SEA will have an opportunity, if necessary, to clarify its plans for peer and staff 
reviewers and to answer any questions reviewers may have during the on-site review. The peer reviewers will then provide comments to the 
Department. Taking those comments into consideration, the Secretary will make a decision regarding each SEA's request for this flexibility. If an 
SEA's request for this flexibility is not granted, reviewers and the Department will provide feedback to the SEA about the components of the SEA's 
request that need additional development in order for the request to be approved. 

This document provides guidance for peer review panels as they evaluate each request during the on-site peer review portion of the review process. 
The document includes the specific information that a request must include and questions to guide reviewers as they evaluate each request. Questions 
that have numbers or letters represent required elements. The italicized questions reflect inquiries that reviewers will use to fully consider all 
aspects of an SEA's plan for meeting each principle, but do not represent required elements. 

In addition to this guidance, reviewers will also use the document titled ESEA Flexibility, including the definitions and timelines, when reviewing each 
SEA's request. As used in the request form and this guidance, the following terms have the definitions set forth in the document titled ESEA 
Flexibility: (1) college- and career-ready standards, (2) focus school, (3) high-quality assessment, (4) priority school, (5) reward school, (6) standards that 
are common to a significant number of States, (7) State network of institutions of higher education, (8) student growth, and (9) turnaround principles. 
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Review Guidance 
Consultation 



1 . Did the SEA meaningfully engage and solicit input on its request from teachers and their representatives? 

^ Is the engagement likely to lead to successful implementation of the SEA 's request due to the input and commitment of teachers and their representatives at the outset of 
the planning and implementation process? 

^ Did the SEA indicate that it modified any aspect of its request based on input from teachers and their representatives? 



Consultation Question 1 Panel Response 

Tally of Veer Responses: 4: Yes 2: No 



Response Component 


Peer Panel Response 


Rationale 


• The narrative cites communications, meetings, and feedback sessions with educator groups, but is not 
specific about how the flexibility request has been informed by these sessions or about actions taken as a 
result of feedback from teachers and their representatives. The letters lacked specificity about the flexibility 
request. 


Strengths 


• The Center for the Teaching Profession (CTP), a unit within the Ohio Department of Education (ODE), is 
focused on human capital improvement and has many mechanisms for communicating with stakeholders, 
including representatives of English Learners and students with disabilities subgroups. CTP is charged with 
this responsibility for consultation. In addition, staff from the Governor's Office conducted 1 8 meetings and 
an email mechanism was created. The request details some modifications that resulted from input. 


Weaknesses, issues, 
lack of clarity 


• While the narrative cites appropriate teacher and educator groups, the nature of the consultation and input as 
well as specific changes made to the request in response to the input is not clear. 


Technical Assistance 
Suggestions 


• Explicitly state what changes were made to the request based on educator input, and include direct outreach 
to teachers, rather than solely through organizations. Moving forward, the SEA should continue its outreach 
efforts and should demonstrate what it has changed based on feedback. 



2. Did the SEA meaningfully engage and solicit input on its request from other diverse communities, such as students, parents, community-based 
organizations, civil rights organizations, organizations representing students with disabilities and English Learners, business organizations, and 
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Indian tribes? 

^ Is the engagement likely to lead to successful implementation of the SEA's request due to the input and commitment of relevant stakeholders at the outset of the planning 
and implementation process? 

^ Did the SEA indicate that it modified any aspect of its request based on stakeholder input? 

"P- Does the input represent feedback from a diverse mix of stakeholders representing various perspectives and interests, including stakeholders from high-need communities? 



Consultation Question 2 Panel Response 

Tally of Peer ResponsesA: Yes 2: No 



Response Component 


Peer Panel Response 


Rationale 


• ODE has formed an internal ESEA Flexibility Committee to seek input and oversee the development of the 
request. The stakeholders cited (pp. 13-14) represent the breadth needed in consideration of the request. 
While the ODE indicated that the feedback was useful, it did not indicate if specific modifications were made 
to the request. 


Strengths 


• ODE described and documented meeting dates with groups representing English Learners and students with 
disabilities, citing some of their concerns with adequate yearly progress (AYP) and testing practices. 
Additionally, charter schools, businesses, and business organizations were given an opportunity to provide 
input. Consultation with English Learners and students with disabilities was particularly strong (p. 15). 


Weaknesses, issues, 
lack of clarity 


• Some peers felt that the narrative fails to be specific on how consultation informed the flexibility request and 
others felt that details were not sufficient about representatives from English Learners and students with 
disabilities representative groups (p. 15). 

• The SEA includes information on some of the concerns voiced by some of the stakeholder groups, such as 
English Learners and charter schools, but does not show how it modified things based on this input (pp. 1 5- 
16). 

• The portal does not seem like a particularly effective communication tool given that it led to only 150-175 
visits and comments out of Ohio's 110,000 teachers and 5,200 administrators. 
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Response Component 


Peer Panel Response 


Technical Assistance 
Suggestions 


• ODE should consider reviewing its protocols for seeking and utilizing input. Additionally, ODE might 
consider continuing to engage representatives from diverse communities during development and 
implementation of activities related to all three principles. Furthermore, ODE should demonstrate that it has 
carefully communicated and tried to address stakeholder concerns in a meaningful way. Finally, ODE should 
consider more in person outreach throughout the State (rather than conference calls). 



Principle 1: College- and Career-Ready Expectations for All Students 



Note to Peers: Staff will review 1 A Adopt ColleseAnd Career-Ready Standards, Options A and B. 



l.B Transition to college- and career-ready standards 



l.B Part A: Is the SEA's plan to transition to and implement college- and career-ready standards statewide in at least reading/language arts and 
mathematics no later than the 2013—2014 school year realistic, of high quality? 

Note to Peers: See ESEA Flexibility Review Guidance for additional considerations related to the types of activities an SEA includes in its transition plan. 



l.B Panel Response, Part A 

Pally of Peer Responses: 0: No 6: Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE has adopted the Common Core State Standards (CCSS) with intent to implement them by 2013-14. 
ODE is in the process of determining the alignment gaps and transition requirements for moving from 
current to new standards. 


Strengths 


• The Ohio legislature has approved bills demonstrating support for the movement to higher standards, 

including bills about "core" graduation requirements and a new college -and career-ready education system in 
English/language arts (ELA), mathematics, science, and social studies. ODE is undertaking an analysis of 
current vs. CCSS standards to prepare crosswalks for schools. Additionally, ODE has a logic model for 
communicating the process to others and for communicating outcomes. 
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Response Component 


Peer Panel Response 


Weaknesses, issues, 
lack of clarity 


• ODE did not report interim progress data, nor was any mention made of efforts to evaluate the success of 
the transition and implementation plan. Furthermore, there was little mention of transition support provided 
for teachers of English Learners and students with disabilities. 


Technical Assistance 
Suggestions 


• Given the number of standards, initiatives, and adoptions, Ohio may profit from continuing to coordinate 
and integrate the components into a single coherent plan to achieve college- and career-ready standards. 
ODE might consider continuing to refine the thinking around how to communicate their overarching reform 
plans as part of its theory of action (p. 22). ODE should develop clear plans to evaluate the success of 
implementation, including reporting interim data. Additionally, ODE should clearly articulate the 
professional development activities of all teachers of students with disabilities and English Learners to 
transition them to college and career level activities. Furthermore, ODE might consider providing more 
information on its plan to integrate standards into teacher and principal preparation programs. For example, 
what sort of evidence is the SEA seeking from teacher preparation programs to ensure that they adequately 
reflect and align with the rigor of CCSS (p. 38)? 



Part B: Is the SEA's plan likely to lead to all students, including English Learners, students with disabilities, and low- achieving students, gaining 
access to and learning content aligned with the college- and career-ready standards? 



Note to Peers: See ESEA Flexibility Review Guidance for additional considerations related to the types of activities an SEA includes in its transition plan. 



l.B Panel Response, Part B 

Tally of Peer Responses: 6 :No :Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE is developing extended content standards based on the CCSS for students with disabilities for 
implementation in the 2013-14 school year. ODE recognizes the need to update English Learners 
proficiency expectations in response to the adoption of the CCSS standards. 


Strengths 


• Ohio has become a participant in the State Collaborative on English Language Acquisition (SCELA). The 
new standards are scheduled for completion in July 2012. Additionally, the career and technical education 
component was strong and addressed the needs of special populations. 
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Weaknesses, issues, 
lack of clarity 


• The narrative on pp. 29 and 30 outlines the current direction, but provides little insight into implementation 
plans. Plans to prepare students and teachers to transition from the alternate assessment based on 
modified academic achievement standards (AA-MAAS) to the new assessment system are not presented. 
Based on review of the request and conversation with ODE staff, it appears that extended standards are 
applicable to all students with disabilities; usually these standards apply only to students with severe 
cognitive disabilities (e.g., those who are taking the alternate assessment based on alternate academic 
achievement standards (AA-AAAS)). 


Technical Assistance 
Suggestions 


• ODE must redefine the student population for its extended standards to students with significant cognitive 
disabilities. ODE should use "people first" language (e.g., the term "students with disabilities" or "people 
with disabilities") instead of "disabled students" here and throughout the proposal. ODE should explain 
why the percentage of students taking AP classes has declined and what steps it is taking to reverse this 
trend. ODE should clarify its plan to transition students and teachers from the AA-MAAS to the new 
standards and assessments. 



l.C Develop and Administer Annual, Statewide, Aligned, High-Quality Assessments that Measure Student Growth 



l.C Did the SEA develop, or does it have a plan to develop, annual, statewide, high-quality assessments, and corresponding academic achievement 
standards, that measure student growth and are aligned with the State's college- and career-ready standards in reading/language arts and 
mathematics, in at least grades 3-8 and at least once in high school, that will be piloted no later than the 2013—2014 school year and planned for 
administration in all LEAs no later than the 2014—2015 school year, as demonstrated through one of the three options below? Does the plan 
include setting academic achievement standards? 

Note to Peers: Staff will review Options A and C. 

If the SEA selected Option B : 

If the SEA is neither participating in a State consortium under the RTTA competition nor has developed and administered high-quality 
assessments, did the SEA provide a realistic, high-quality plan describing activities that are likely to lead to the development of such 
assessments, their piloting no later than the 2013—2014 school year, and their annual administration in all LEAs beginning no later than the 
2014—2015 school year? Does the plan include setting academic achievement standards? 
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l.C, Option B Panel Response 

1X1 Not applicable because the SEA selected 1.C, Option A or Option C 
Tally of Peer Responses: X Yes, X No 



Response Component 


Peer Panel Response 


Rationale 




Strengths 




Weaknesses, issues, 
lack of clarity 




Technical Assistance 
Suggestions 





Principle 1 Overall Review 



Is the SEA's plan for transitioning to and implementing college-and career-ready standards, and developing and administering annual, statewide, aligned 
high-quality assessments that measure student growth, comprehensive, coherent, and likely to increase the quality of instruction for students and 
improve student achievement? If not, what aspects are not addressed or need to be improved upon? 



Principle 1 Overall Review Panel Response 

Tally of Peer Responses: 0: Yes 6: No 



Response Component 


Peer Panel Response 


Rationale 


• ODE has demonstrated a comprehensive plan for transition and implementation by 2013-2014 and has considered 
subgroup needs and performance throughout the planning process. However, it is of great concern that the extended 
standards are applicable to all students with disabilities, which limits educational opportunities for the vast majority of 
students with disabilities. Additionally, there are no plans to transition from the AA-MAAS to the general assessment. 
Although ODE selected both Option A and Option B, Attachment 5 was not included in its initial submission. 


Strengths 


• ODE uses a theory of action to provide a basis for understanding the transition to college- and career-ready standards 
and assessments, as well as promising, evidence-based best practices including formative assessments and dual 
enrollment. The SEA is also taking steps to ensure that all students have access to a college- and career-ready 
curriculum by expanding dual enrollment and AP offerings and expanding CTE to align with CCSS standards. ODE 
uses a transition timeline that provides for full transition, including assessments, by 2014-2015. Students with 
disabilities and English Learners subgroups are considered throughout the transition planning phases. 
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Response Component 


Peer Panel Response 


Weaknesses, issues, 
lack of clarity 


• It is unclear how the SEA plans to transition its current assessments, including the AA-MAAS, over to the assessment 
being developed by the Partnership for Assessment of Readiness for College and Careers (PARCC) by 2014-2015. 
Additionally, it is problematic if all students with disabilities are participating in the extended standards system. 

• The percentage of students taking AP exams has declined by more than 12 percent; however, the request does not 
address this and cites it as a benefit with respect to increasing the number of underrepresented students among all AP 
students. It is also unclear to what extent increasing the numbers of students in AP classes is aligned with success in 
AP exams (p. 36). Some peers felt that there was a lack of educational soundness in using a "3" as a passing grade on 
an AP exam. 


Technical Assistance 
Suggestions 


• Given the number of standards, initiatives, and adoptions, Ohio may profit from bringing together the components, 
and defining the new direction, in a single coherent plan to achieve college- and career-ready standards. ODE might 
consider continuing to refine the thinking around how to communicate their overarching reform plan as part of its 
theory of action (p. 22). ODE might consider developing clear plans to evaluate the success of implementation, 
including reporting interim data. ODE should demonstrate increased commitment to low-income populations 
through professional development, instructional materials, and resources. Additionally, ODE should clearly articulate 
the professional development activities of all teachers of students with disabilities and English Learners to transition 
them to college- and career-level activities. ODE should re-examine its decision to have extended standards to apply 
to all students with disabilities. Additionally, ODE should develop a plan to transition from the AA-MAAS to the 
general assessment. Furthermore, ODE might consider providing more information on its plan to integrate standards 
into teacher and principal preparation programs. For example, what sort of evidence is the SEA seeking from teacher 
preparation programs to ensure that they adequately reflect and align with the rigor of CCSS (p. 38)? ODE should use 
the term "students with disabilities" or "people with disabilities" instead of "disabled students". ODE should explain 
why the percent of students taking AP classes has declined and what it is doing to reverse this trend. 
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Principle 2: State-Developed Differentiated Recognition, Accountability, and Support 
2.A Develop and Implement a State-Based System of Differentiated Recognition, Accountability, and Support 



2.A.i Did the SEA propose a differentiated recognition, accountability, and support system, and a high-quality plan to implement this system no later 
than the 2012—2013 school year, that is likely to improve student achievement and school performance, close achievement gaps, and increase the quality 
of instruction for students? { note to Veers, please write to this question after completing 2.A.i.a and 2.A.i.b) 



2A.i Panel Response 

Tally of Peer Responses: 6:No 0:Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE recognizes the need to unify its approach to accountability and the need to make the system 
understandable. To accomplish this, ODE proposes a letter grade system (F- A) to replace the six 
descriptive labels ranging from Academic Emergency to Excellent with Distinction. The new system will 
add several new performance measures, but will retain the Education Value Added Assessment System 
(EVAAS) value-added model. While subgroup performance is an important determination of letter grade 
designation, it is unclear what steps are being taken to ensure that all students and subgroups are represented 
in the accountability system. 


Strengths 


• ODE's plan includes differentiated recognition that recognizes excellence in several ways, as well as 
identifying problems, and differentiated levels of support. ODE's accountability system utilizes both 
formative and summative information and includes information from different levels in the State, including 
LEAs, schools, administrators, teachers, and students. ODE presented a clear method of calculating each 
letter grade. ODE's proposed additional measures are innovative and could have broad utility to other 
States. 
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Response Component 


Peer Panel Response 


Weaknesses, issues, 
lack of clarity 


• ODE's differentiated levels of support did not include much detail. Further thought may be needed as to 
how additional measures are calculated, including graduation rate. It is unclear to what extent value-added 
analysis adds to the transparency of the system. There are inconsistencies in the request with respect to 
which subgroups are disaggregated for reporting and accountability purposes. Additionally, it may be 
challenging to communicate the new system to ensure transparent reporting to educators and the public. It 
may be problematic that an insufficient participation rate discounts a letter grade in only the achievement and 
graduation gap performance measures — in other words, the participation rate is important and the 
consequence for noncompliance should be more severe. It is not clear that graduation rate constitutes a 
significant factor in the accountability system for all secondary schools. 


Technical Assistance 
Suggestions 


• ODE might consider providing further information on the differentiated levels of support that will be 
provided to districts. ODE should integrate the criteria for identifying focus and priority schools into 
computation of graduation rate in the overall accountability system. ODE should consider the development 
of standards-based definitions of what it means to earn each individual letter grade within this system. These 
definitions should be clearly communicated to educators, parents, students, relevant stakeholders, and the 
public. ODE should include all ESEA subgroups for reporting and accountability purposes. ODE should 
provide the rationale used to establish its "n size" at 30 and ensure that all students are included in the 
accountability system some way. ODE should provide ongoing monitoring and evaluation of the 
implementation of its accountability system. ODE should clearly articulate how graduation rate factors into 
the overall accountability score for secondary schools. ODE should ensure that participation rate is 
sufficiently accounted for in computation of individual measures and overall score in a way that promotes 
high levels of participation and should consider setting an absolute floor, or minimum level, that is 
acceptable. 
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a. Does the SEA's accountability system provide differentiated recognition, accountability, and support for all LEAs in the State and for all 
Title I schools in those LEAs based on (1) student achievement in reading/language arts and mathematics, and other subjects at the State's 
discretion, for all students and all subgroups of students identified in ESEA section llll(b)(2)(C)(v)(H); (2) graduation rates for all 
students and all subgroups; and (3) school performance and progress over time, including the performance and progress of all subgroups? 



2.A.i.a Panel Response 

Tally of Peer Responses : 6: No 0:Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE's strategy categorizes school performance in relation to annual measurable objectives (AMOs) 
with a letter grade from A to F. 

• This rating system will be applied to both content areas and graduation rates. The same letter grade is 
applied to school performance and progress over time, including the performance of subgroups. 
There is an attempt to increase transparency in ODE's new system and add additional measures over 
time. 


Strengths 


• Table 2, Overall Cumulative Letter Grade Designations, provides a clear definition of the calculation 
criteria for assigning letter grades. The continued use of value-added analysis could be useful as part of 
an overall accountability system. Additionally, ODE tried to add transparency to student performance. 
ODE also used real student data and provided additional indicators, including: readiness, gifted 
performance, fiscal performance, non-academic performance, and measures of rigor (p. 49-50). ODE 
also plans to create a new index to identify the largest achievement gaps. 


Weaknesses, issues, lack of 
clarity 


• It is unclear whether the use of value-added analysis adds to transparency. ODE's plan is dependent 
upon legislative action. 


Technical Assistance 
Suggestions 


• ODE should explain how it will ensure that the continued use of value-added analysis does not hinder 
its efforts to improve transparency around student performance. ODE should consider the 
development of standards-based definitions of what it means to earn each individual letter grade within 
this system. These definitions should be clearly communicated to educators, parents, students, 
relevant stakeholders, and the public. ODE should monitor carefully to ensure that the new system 
does represent a unified and transparent approach to accountability. 



b. Does the SEA's differentiated recognition, accountability, and support system create incentives and provide support that is likely to be 
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effective in closing achievement gaps for all subgroups of students? 



2.A.i.b Panel Response 

Tally of Peer Responses: 6: No 0: Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE is proposing a unified system that aims to be more transparent and includes several innovative 
measures of performance and achievement. It is not yet clear how subgroups are included for reporting 
and accountability purposes in a way that ensures accountability for all students. 


Strengths 


• Graphics 1-8 (pp. 54-59) demonstrate trends in subgroup performance gaps, including students with 
disabilities and English Learners, under the current system data release (2007—2011). The growth model 
EVAAS will still be a safe harbor option, but the highest rewards will still be reserved for the groups 
that make expected academic progress in the current year. ODE's commitment to revisiting its business 
rules represents a best practice for ongoing development of the accountability system. Additionally, 
subgroup performance is an important determinant in final letter grade status (p. 68). 


Weaknesses, issues, 
lack of clarity 


• On page 64 inconsistencies exist between the data provided in the graphs and the text. Furthermore, 
there are problems in the graduation rate calculations on page 67 that need to be reconciled. There are 
inconsistencies in the request with respect to which subgroups are disaggregated for reporting and 
accountability purposes. (See page 72 for an example). Additionally, it may be challenging to 
communicate the new system to ensure transparent reporting to educators, parents, and the public. 


Technical Assistance 
Suggestions 


• ODE should clarify how English Learner students in their first two years of enrollment will be included 
in the accountability system across content areas (p. 59). Consider what additional status criteria is 
required to reconcile the graduation rate table on page 67. Clarify specifically which subgroups will be 
considered in the final subgroup analysis; ODE should include all ESEA subgroups for reporting and 
accountability purposes. ODE should provide the rationale used to establish its "n size" at 30 and 
ensure that all students are included in the accountability system some way. ODE should provide 
ongoing monitoring and evaluation of the implementation of its accountability system. ODE should 
provide further clarity around continued progress over time in districts that achieve independent status 
in order to be certain that all students are achieving at an acceptable level. 



c. Note to Peers: Staff will review 2.JLi.c 
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Note to Veers: Staff will review 2A.H Option A. 

ONLY FOR SEAs SELECTING OPTION B : If the SEA elects to include student achievement on assessments other than 
reading/language arts and mathematics in its differentiated recognition, accountability, and support system by selecting Option B, review 
and respond to peer review questions in section 2.A.ii. If the SEA does not include other assessments, go to section 2.A.iii. 

2.A.ii Did the SEA include student achievement on assessments in addition to reading/language arts and mathematics in its differentiated recognition, 
accountability, and support system and to identify reward, priority, and focus schools? 

a. Note to Peers: Staff will review 2.A.ii.a 

b. Does the SEA's weighting of the included assessments result in holding schools accountable for ensuring all students achieve the State's 
college- and career-ready standards? 

c. Note to Peers: Staff will review 2 A.ii.c 

2.A.ii.b Panel Response 

1X1 Not applicable because the SEA selected 2 A, Option A 

Tally of Veer Responses: X Yes, X No 



Response Component 


Veer Panel Response 


Rationale 


Selected Option A (p. 52). However, the Percentage of State Indicators Met (p. 46) includes reading, writing, 
mathematics, social studies, and science. An explanation of the inconsistency is needed. 


Strengths 




Weaknesses, issues, 
lack of clarity 




Technical Assistance 
Suggestions 





2.B Set Ambitious but Achievable Annual Measurable Objectives 



2.B Note to Peers: Staff will review Options A and B. 

Did the SEA describe the method it will use to set new ambitious but achievable annual measurable objectives (AMOs) in at least 
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reading/language arts and mathematics, for the State and all LEAs, schools, and subgroups, that provide meaningful goals and are used to 
guide support and improvement efforts through one of the three options below? 

If the SEA selected Option C : 

Did the SEA describe another method that is educationally sound and results in ambitious but achievable AMOs for all LEAs, schools, and 
subgroups? 

i. Did the SEA provide the new AMOs and the method used to set these AMOs? 

ii. Did the SEA provide an educationally sound rationale for the pattern of academic progress reflected in the new AMOs? 

iii. If the SEA set AMOs that differ by LEA, school, or subgroup, do the AMOs require LEAs, schools, and subgroups that are further 
behind to make greater rates of annual progress? 

iv. Did the SEA attach a copy of the average statewide proficiency based on assessments administered in the 2010—2011 school year in 
reading/language arts and mathematics for the "all students" group and all subgroups? (Attachment 8) 

^ Are these AMOs similarly ambitious to the AMOs that would result from using Option A orB above? 

Are these AMOs ambitious but achievable given the State's existing proficiency rates and any other relevant circumstances in the State? 

Will these AMOs result in a significant number of children being on track to be college- and career-ready? 
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2.B, Option C (including Questions i-iv) Panel Response 

I | Not applicable because the SEA selected 23, Option A or Option B 
Tally of Peer Responses: 6:Yes 0: No 



Response Component 


Peer Panel Response 


Rationale 


• ODE provides extensive analysis and honest interpretation of trend data on achievement gaps, including data 
from the State's alternative assessment. The mechanisms for setting AMOs in mathematics and reading and 
for setting graduation targets are described in detail and defensible. ODE has set a goal of reducing by half 
the percentage of all students groups that are not proficient within six years. ODE described the 
methodology for computing state AMO targets, and then included a chart showing needed progress to 
achieve the new targets through 2016-17. All subgroups are to be evaluated against the AMO for all students 
(p. 60). AMOs are set in equal intervals until reaching the target. 


Strengths 


• Plans for transitioning to new assessments are presented. Letter grade designations are easy to understand 

onn \T7pl 1 ri f 1 "H t~i f 1 ri /I 1 1 1 ria tct rnf Hie 1~-t — i r-fe tret nlflnnol txti rilir* c rnnn c a t~i ri rntntnn mtiT c r* ri f^ol c a vp f* c F 1 fF*rH a t~i ri 
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nicely illustrate the feasibility and utility of the system. 

• Table 8 (p. 60) clearly shows the baseline for the new AMO targets and the needed progression. Using the 

mpfnnn nplmp^itpn on r>^iO"P nf) ^rnnnk with nwpt nvofipipnpv f^itp^ won n h^ivp to tn^lrp nicxTPV o^ms to vp^ph 

the state targets. 

• ODE included a chart showing average proficiency in reading and mathematics for all students, including 
subgroups (Att. 8, pp. 161-166). 


Weaknesses, issues, lack 
of clarity 


• There is concern that English Learner students are not adequately included during their first two years of 
enrollment in a school in the United States in the reading assessment, in contrast to ESEA regulations that 
focuses on English Learners in their first year of enrollment in a school in the U.S. 


Technical Assistance 
Suggestions 


• ODE should clarify how it will ensure that students who are English Learners will be included in the 
accountability system in their first two years of enrollment in a school in the United States. ODE might 
consider setting subgroup goals to further ensure that subgroup performance is accounted for and 
represented. 
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2.C Reward Schools 



2.C.i Did the SEA describe its methodology for identifying highest-performing and high-progress schools as reward schools? 



2.C.i Panel Response 

Tally of Peer Responses: NA 



Response Component 


Peer Panel Response 


Rationale 


• ODE proposes a system of determining reward schools that meets the ESEA flexibility definition that 
includes both Title I and Title I-eligible schools. Furthermore, it builds upon a system of support and 
recognition that ODE has had in place for a number of years. 


Strengths 


• ODE's "Promise Schools" are recognized beyond ODE. ODE proposes to use reward schools as model 
sites. 


Weaknesses, issues, lack 
of clarity 


• It may be a challenge to communicate to parents and the public the differences between the current methods 
and the new methods of determining reward schools. 


Technical Assistance 


• ODE should carefully consider communication plans to articulate the distinctions between various reward 
school classifications. ODE should consider creating a comprehensive plan to create more unity among its 
Reward school designations. 



Note to Peers: Staff will review 2.C.H. 
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2.C.iii Are the recognition and, if applicable rewards, proposed by the SEA for its highest-performing and high-progress schools likely to be 
considered meaningful by the schools? 

^ Has the SEA consulted with J .E As and schools in designing its recognition and, where applicable, rewards? 



2.C.iii Panel Response 

Tally of Veer Responses: 6: No 0: Yes 



Response Component 


Ti Ti 1 T) j_ 

Peer Panel Kesponse 


Rationale 


• ODE has proposed a system of public recognition; however, there is no evidence that the rewards are 

meaningful. Additionally, ODE should clarify the relationship between current reward school designations 
and the ESEA flexibility reward school designation for purposes of public understanding. 


Strengths 


• ODE has a system of public recognition for schools including news releases and recognition at conferences 
(p. 73). ODE intends to collect and share best practices and put them on the SEA's website. 


Weaknesses, issues, lack 
of clarity 


• It does not appear that stakeholder input influenced the selection and design of reward mechanisms and no 
additional evidence was provided to indicate that the rewards were meaningful. ODE does not cite a 
financial reward system. While it will be useful to hold up these schools as examples, putting information on 
a website may not be sufficient. 


Technical Assistance 
Suggestions 


• ODE might consider clarifying the relationship between the current reward school designations and the 
ESEA flexibility reward school designation for purposes of public understanding. ODE should provide 
other ways of providing best practice sharing across schools throughout the State. ODE should consider 
financial rewards or additional flexibility options for reward schools. 
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1 


2.D Priority Schools | 



Note to Veers: Staff will review 2.D.i and 2.D.H. 



2.D.iii Are the interventions that the SEA described aligned with the turnaround principles and are they likely to result in dramatic, systemic change in 
priority schools? 

a. Do the SEA's interventions include all of the following? 

(i) providing strong leadership by: (1) reviewing the performance of the current principal; (2) either replacing the principal if such a 
change is necessary to ensure strong and effective leadership, or demonstrating to the SEA that the current principal has a track record 
in improving achievement and has the ability to lead the turnaround effort; and (3) providing the principal with operational flexibility in 
the areas of scheduling, staff, curriculum, and budget; 

(ii) ensuring that teachers are effective and able to improve instruction by: (1) reviewing the quality of all staff and retaining only those 
who are determined to be effective and have the ability to be successful in the turnaround effort; (2) preventing ineffective teachers 
from transferring to these schools; and (3) providing job-embedded, ongoing professional development informed by the teacher 
evaluation and support systems and tied to teacher and student needs; 

(iii) redesigning the school day, week, or year to include additional time for student learning and teacher collaboration; 

(iv) strengthening the school's instructional program based on student needs and ensuring that the instructional program is research-based, 
rigorous, and aligned with State academic content standards; 

(v) using data to inform instruction and for continuous improvement, including by providing time for collaboration on the use of data; 

(vi) establishing a school environment that improves school safety and discipline and addressing other non-academic factors that impact 
student achievement, such as students' social, emotional, and health needs; and 

(vii) providing ongoing mechanisms for family and community engagement? 
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2.D.iii.a (including questions (i)-(vii)) Panel Response 

Tally of Peer Responses: 2: No 4: Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE's plan is generally aligned with the turnaround principles; however, the plan for support and monitoring 
of implementation is lacking sufficient detail. 


Strengths 


• ODE has a clear set of actions for intervention in priority schools. For those schools that are not being 
served through the School Improvement Grants (SIG) program, there is a choice of using the Ohio 
Intervention and Improvement (Oil) model that provides a somewhat more flexible but rigorous intervention 
(Table 17, p. 78). Additionally, ODE's proposed method for identifying priority schools actually identifies a 
group slightly larger than five percent of the State's Title I schools. The intervention methods are sound and 
will be required for three years. 


Weaknesses, issues, 
lack of clarity 


• ODE presented no plans for supporting and monitoring implementation at the local school level. 


Technical Assistance 
Suggestions 


• ODE should implement monitoring and support mechanisms for ensuring fidelity of implementation. ODE 
should also consider carefully how detailed information and expectations for each turnaround model will be 
communicated with schools and districts. ODE should provide more detail regarding how it will be 
determined whether a principal has the capacity to turn around a school in its Oil model and who will make 
this determination. 



b. Are the identified interventions to be implemented in priority schools likely to — 

(i) increase the quality of instruction in priority schools; 

(ii) improve the effectiveness of the leadership and the teaching in these schools; and 

(iii) improve student achievement and, where applicable, graduation rates for all students, including English Learners, students with 
disabilities, and the lowest-achieving students? 
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2.D.iii.b (including questions (i)-(iii)) Panel Response 

Tally of Peer Responses: 4: No 2: Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE's intervention strategies are sound on face value but appear to lack meaningful state -level support and 
do not include specific information for addressing the needs of English Learners and students with disabilities. 


Strengths 


• Eligible schools will be able to compete for SIG funding; to date, 85 schools have received SIG funding. 
Additionally, there are plans to provide technical assistance and additional supports to priority schools. 
ODE's proposed interventions are aligned with turnaround principles. 


Weaknesses, issues, lack 
of clarity 


• Although SIG implementation began in 2010, no data were provided on the progress of SIG schools thus far. 
The plan appears to lack meaningful state-level support and does not include specific information for 
addressing the needs of English Learners and students with disabilities. It is unclear whether schools that are 
currently SIG schools will have any additional supports from ODE. It is unclear who will provide support to 
the schools. 


Technical Assistance 
Suggestions 


• ODE should provide data on schools currently involved in the improvement process. ODE should define its 
state-level support system and clarify who is going to be responsible for implementation and support. ODE 
should describe its plan for monitoring fidelity of implementation of its plan at the LEA and school level. 
ODE should consider and articulate the role of the Regional Education Support Centers (RESCs) in 
supporting priority schools. 



c. Note to Peers: Staff will review 2.D.iii.c 



2.D.iv Does the SEA's proposed timeline ensure that LEAs that have one or more priority schools will implement meaningful interventions aligned 
with the turnaround principles in each priority school no later than the 2014—2015 school year? 

^ Does the SEA 's proposed timeline distribute priority schools' implementation of meaningful interventions aligned with the turnaround principles in a balanced way, 
such that there is not a concentration of these schools in the later years of the timeline? 
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2.D.iv Panel Response 

Tally of Peer Responses: 0: No 6: Yes 



Response Component 


Peer Panel Response 


Rationale 


• KjLJii, b uiiiciiric icc^uiich ncwiy lrcipicrcicnnrig priority bcnooib to Dcgin liiipicriicntatiori or lntcivcntiorib in 
the 2013-2014 school year (pp. 78-79). 


Strengths 


• ODF trf*ovinf*n a Hmplinp and nlarm to nfnviHp additional tprhniral asmstanrp and fpsonfrps to nffofitv 

schools. 


Weaknesses, issues, lack 
of clarity 


• It is not possible from the information on page 79 to determine whether or not there will be a concentration 
of identified schools at the end of the three year-period. 


Technical Assistance 
Suggestions 


• ODE could further define and be more explicit about the technical assistance and support it intends to 
provide to priority schools. 



2.D.V Did the SEA provide criteria to determine when a school that is making significant progress in improving student achievement exits priority 
status? 

a. Do the SEA's criteria ensure that schools that exit priority status have made significant progress in improving student achievement? 

^ Is the level of progress required by the criteria to exit priority status likely to result in sustained improvement in these schools? 



2.D.V and 2.D.v.a PANEL RESPONSE 

Tally of Peer Responses: 6: No 0: Yes 



Response Component 


Peer Panel Response 


Rationale 


• The exit criteria and timeline remain unclear. 


Strengths 


• ODE attempts to tie specific exit criteria to the initial method for identifying schools in priority status. ODE 
requires schools in priority school status to make sustained progress over several years; schools that fail to do 
so will be identified as persistently low-achieving. 


Weaknesses, issues, 
lack of clarity 


• ODE's definition of exit criteria is minimal and insufficient. 
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Response Component 


Peer Panel Response 


Technical Assistance 
Suggestions 


• ODE should better clarify the timeline and criteria for exiting priority status, including: whether schools exit 
only after three years, and whether this lowest-performing five percent criterion used to identify priority 
schools is tied to a specific benchmark standard (p. 79). ODE should provide technical assistance 
surrounding priority school exit criteria. 



2.E Focus Schools 



Note to Peers: Staff will review 2.E.i, 2.E.i.a, and 2.E.H 

2.E.i Did the SEA describe its methodology for identifying a number of low-performing schools equal to at least 10 percent of the State's Title I 
schools as focus schools? If the SEA's methodology is not based on the definition of focus schools in ESEA Flexibility (but is instead, e.g., 
based on school grades or ratings that take into account a number of factors), did the SEA also demonstrate that the list provided in Table 2 is 
consistent with the definition, per the Department's "Demonstrating that an SEA's Lists of Schools Meet ESEA Flexibility Definitions" 
guidance? 

a. Note to Peers: Staff will review 2.E.i.a. 

b. Is the SEA's methodology for identifying focus schools educationally sound and likely to ensure that schools are accountable for the 
performance of subgroups of students? 

2.E.i.b Panel Response 



Tally of Peer Responses: 0: No 6: Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE's methodology for identifying and monitoring focus schools is sound and includes subgroup metrics. 


Strengths 


• ODE's submission included Table 18, Subgroup Proficiency and Graduation 85th Percentile Gaps (p. 81). 
ODE's plan includes subgroup status in the final calculation. Additionally, a school may meet its AMO, but 
still be designated as a focus school based on subgroup performance. 


Weaknesses, issues, 
lack of clarity 


• It is not clear whether ODE's subgroups are mutually exclusive. 
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Response Component 


Peer Panel Response 


Technical Assistance 
Suggestions 


• ODE should clarify how subgroups are defined and whether subgroups are mutually exclusive. 



2.E.ii Note to Peers: Staff will review 2.E.H 

2.E.iii Does the SEA's process and timeline ensure that each LEA will identify the needs of its focus schools and their students and implement 
interventions in focus schools at the start of the 2012—2013 school year? Did the SEA provide examples of and justifications for the 
interventions the SEA will require its focus schools to implement? Are those interventions based on the needs of students and likely to 
improve the performance of low-performing students and reduce achievement gaps among subgroups, including English Learners and students 
with disabilities? 

^ Has the SUA demonstrated that the interventions it has identified are effective at increasing student achievement in schools with similar characteristics, needs, and 
challenges as the schools the SEA has identified as focus schools? 

^ Has the SEA identified interventions that are appropriate for different levels of schools (elementary, middle, high) and that address different types of school needs 
(e.g., all-students, targeted at the lowest-achieving students)? 

2.E.iii Panel Response 

Tally of Peer Responses: 5: No 1: Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE will publish the focus school identification list in August 2012 based on school year (SY) 201 1-2012 

performance results. Schools will be required to implement the Ohio Improvement Model with oversight from 
their LEA and regional SEA during the first semester of SY 2012-2013. It is unclear whether this timeline will 
enable LEAs to spend sufficient time diagnosing and preparing for implementation to occur during the first 
semester of SY 2012-2013. 


Strengths 


• ODE's monitoring team will selectively check implementation of the interventions. 


Weaknesses, issues, 
lack of clarity 


• ODE makes no mention of English Learners in this section of its request and there are no specific plans for 
how the needs of students with disabilities will be addressed in the process of implementing interventions in 
focus schools. Since ODE is not planning to identify schools as focus schools until August 2012, it is not clear 
that schools will be able to implement instructionally-focused interventions in the first semester. 
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Response Component 


Peer Panel Response 


Technical Assistance 
Suggestions 


• ODE should describe the supports that will be available to ensure that the needs of students with disabilities 
and English Learners will be addressed in the planning process. ODE should reconsider its proposed timeline 
for focus school identification to allow for implementation of focus school interventions by the first semester 
of SY 2012-2013. ODE might further consider adjusting its testing timeline or take other actions designed to 
shorten the timeline between test administration and data availability; however, ODE should be cautious of the 
potential this creates for a situation where schools are publically identified, given that much of this information 
will not be verifiable until the data are finalized. Some peers felt that ODE might further consider providing 
early notice to schools for early identification purposes; however, other peers expressed concern with this 
suggestion. 



2.E.iv Did the SEA provide criteria to determine when a school that is making significant progress in improving student achievement and narrowing 
achievement gaps exits focus status? 

a. Do the SEA's criteria ensure that schools that exit focus status have made significant progress in improving student achievement and 
narrowing achievement gaps? 

^ Is the level of progress required by the criteria to exit focus status likely to result in sustained improvement in these schools? 



2.E.iv and 2.E.iv.a PANEL RESPONSE 

Tally of Peer Responses: 6: No 0: Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE attempts to tie its focus school exit criteria to the initial selection criteria; however, the timeline and 
exit criteria are insufficiently defined. 


Strengths 


• ODE attempts to align its exit criteria to its initial selection methodology. 


Weaknesses, issues, 
lack of clarity 


• There is limited discussion and definition of exit criteria and timeline. 
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Peer Panel Re show se 


Technical Assistance 
Suggestions 


• ODE should better clarify the timeline and criteria for exiting focus status, including: the window of time 
schools have to demonstrate sufficient progress and whether this exit criterion is tied to a specific benchmark 
standard (p. 83). ODE should provide technical assistance surrounding focus school exit criteria. ODE 
should clarify key terms, including "enough". 



2.F Provide Incentives and Support for other Title I Schools 



2.F.i Does the SEA's differentiated recognition, accountability, and support system provide incentives and supports for other Title I schools that, 
based on the SEA's new AMOs and other measures, are not making progress in improving student achievement and narrowing achievement 
gaps? 



2.F.i Panel Response 

Tally of Peer Responses: 5: No V.Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE's system of incentives and supports will help to address gap goals and outcomes through data driven 
analysis and planning; however, it is unclear whether these efforts will be enough to improve student 
outcomes for all students, particularly students with disabilities and English Learners. It is not clear how 
AMOs will be used to identify interventions. 


Strengths 


• The tables show application at the district level; however, an individual school earning an "F" will trigger 
movement of the district to high support. ODE has indicated how it intends to provide support for nearly 
30 percent of LEAs. ODE intends to extend support for its community schools as well as for its traditional 
schools. 


Weaknesses, issues, 
lack of clarity 


• It is unclear how AMOs will be used to identify specific interventions for schools or districts. ODE 

proposes a district-level intervention strategy rather than a school-level intervention strategy for providing 
differentiated support and intervention. 
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Response Component 


Peer Panel Response 


Technical Assistance 
Suggestions 


• ODE should explain how AMOs and other performance data will be used to provide different levels of 
support for districts and schools. ODE should consider a monitoring and feedback structure for guiding 
schools through its system of supports to ensure that, where needed, schools and districts are implementing 
appropriate interventions. 



2.F.ii Are those incentives and supports likely to improve student achievement, close achievement gaps, and increase the quality of instruction for all 
students, including English Learners and students with disabilities? 



2.F.ii Panel Response 

Tally of Peer Responses: 6: No 0: Yes 



Response Component 


Peer Panel Response 


Rationale 


• It is not clear from the information provided that the incentives and supports are likely to improve student 
achievement, particularly for low-achieving students, English Learners, and students with disabilities. 


Strengths 


• ODE will provide assessments and diagnostic tools that should assist LEAs in efforts to address subgroup 
performance. 


Weaknesses, issues, lack 
of clarity 


• No evidence is provided as to how the needs of students with disabilities and English Learners will be 

considered in the process of implementing interventions in focus schools. ODE's focus on districts rather 
than schools necessitates some mechanism to hold districts accountable for individual school performance. 
Some peers felt that the outcomes that are stated in the theory of action are not evident in this section. 


Technical Assistance 
Suggestions 


• ODE should provide evidence that the system as proposed will result in the desired outcomes. ODE 

should develop a clearer method for holding districts accountable for individual school performance. ODE 
might think about using the outcomes in its theory of action to organize the SEA's plan to provide support 
and interventions to other Title I schools. 
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2.G Build SEA, LEA, and School Capacity to Improve Student Learning 



2.G Is the SEA's process for building SEA, LEA, and school capacity to improve student learning in all schools and, in particular, in low- 
performing schools and schools with the largest achievement gaps, likely to succeed in improving such capacity? 

i. Is the SEA's process for ensuring timely and comprehensive monitoring of, and technical assistance for, LEA implementation of 
interventions in priority and focus schools likely to result in successful implementation of these interventions and in progress on leading 
indicators and student outcomes in these schools? 

^ Did the SEA describe a process for the rigorous review and approval of any external providers used by the SEA and its EEAs to support the implementation 
of interventions in priority and focus schools that is likely to result in the identification of high-quality partners with experience and expertise applicable to the 
needs of the school, including specific subgroup needs? 

ii. Is the SEA's process for ensuring sufficient support for implementation of interventions in priority schools, focus schools, and other Title 
I schools under the SEA's differentiated recognition, accountability, and support system (including through leveraging funds the LEA was 
previously required to reserve under ESEA section 11 16(b) (10), SIG funds, and other Federal funds, as permitted, along with State and 
local resources) likely to result in successful implementation of such interventions and improved student achievement? 

iii. Is the SEA's process for holding LEAs accountable for improving school and student performance, particularly for turning around their 
priority schools, likely to improve LEA capacity to support school improvement? 



2.G (including i, ii, and iii) Panel Response 

Tally of Peer Responses: 6: No 0: Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE has reorganized to provide support for its lowest-performing schools. The list of teams, 
processes, and programs is impressive, but the narrative does not clarify how these efforts are tied 
together to achieve the desired improvements in student achievement. ODE attests to having a 
rigorous competitive process for selection and review for providers, but does not provide detail. 


Strengths 


• Peers felt that the extent to which the SEA organizes around this challenge will determine the level of 
success in supporting schools to improve performance and close performance gaps. Additionally, 
ODE provides an extensive list of resources available to help build capacity. 
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Response Component 


Peer Panel Response 


Weaknesses, issues, lack of 
clarity 


• It is unclear to what extent ODE's list of services and supports fits together in a comprehensive 
manner. It is unclear how Title I or other funds will be used to promote improvement. Specific 
services related to students with disabilities and English Learners are not identified. ODE does not 
provide a timeline, sequencing, and/ or a clearly articulated plan for the monitoring and 
implementation of intervention and support strategies. 


Technical Assistance Suggestions 


• ODE should distinguish services and supports for LEAs based on the individual LEA needs. ODE 
should further consider how to meet the needs of English Learners and students with disabilities. 
ODE should provide a timeline, sequencing, and/ or a clearly articulated plan for the monitoring and 
implementation of intervention and support strategies. 



Principle 2 Overall Review 



Is the SEA's plan for developing and implementing a system of differentiated recognition, accountability, and support likely to improve student 
achievement, close achievement gaps, and improve the quality of instruction for students? Do the components of the SEA's plan fit together to create 
a coherent and comprehensive system that supports continuous improvement and is tailored to the needs of the State, its LEAs, its schools, and its 
students? If not, what aspects are not addressed or need to be improved upon? 



Principle 2 Overall Review Panel Response 

Tally of Peer Responses: 6: No 0: Yes 



Response Component 


Peer Panel Response 


Rationale 


• ODE suggests the need to create a unified plan, but does not in the narrative show a clear path to achieving that 
unity. The plan does promote an improved use of data and indicators to measure school performance and progress. 
However, the SEA organization and supports seem to lack the recognized need for unity. The overall plan seems to 
miss the opportunity to clarify and focus school improvement efforts. 


Strengths 


• ODE's plan includes differentiated recognition that recognizes excellence in several ways, as well as identifying 
problems and differentiated levels of support. Nearly 35 percent of LEAs will receive some level of support and 
intervention from ODE. 


Weaknesses, issues, lack 
of clarity 


• ODE did not demonstrate how it is evaluating current interventions and creating new ones to achieve desired 
outcomes. ODE's current proposal is neither comprehensive nor coherent. 
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Response Component 


Peer Panel Response 


Technical Assistance 
Suggestions 


• ODE, LEA, and school personnel should be clear about the policy and the specific activity /program goals. ODE 
should articulate a plan, tied to its theory of action, which allows it to achieve its desired goals. 



Principle 3: Supporting Effective Instruction and Leadership 



3.A Develop and Adopt Guidelines for Local Teacher and Principal Evaluation and Support Systems 



3.A.i Has the SEA developed and adopted guidelines consistent with Principle 3 through one of the two options below? 
If the SEA selected Option A : 

If the SEA has not already developed and adopted all of the guidelines consistent with Principle 3: 

i. Is the SEA's plan for developing and adopting guidelines for local teacher and principal evaluation and support systems likely to result 
in successful adoption of those guidelines by the end of the 201 1-2012 school year? 



3 A.i, Option A.i Panel Response 

I | Not applicable because the SEA selected 3. A, Option B 
Tally of Peer Responses: 6: Yes 0: No 



Response Component 


Peer Panel Response 


Rationale 


• The SEA's plan for developing and adopting guidelines for local teacher and principal evaluation 
and support systems is likely to result in successful adoption of those guidelines by the end of SY 
2011—2012 in part because the work in this area will be a continuation of Ohio's demonstrated 
commitment to teacher effectiveness as evidenced by a series of actions since 2005, including 
passage of HB 153. The lofty goals included in the State's Race to the Top application remain the 
foundation for the Ohio's efforts to improve in this area. 
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Response Component 


Peer Panel Response 


Strengths 


• The guidelines are formally being adopted in July 2012 and the implementation timeline (as 

described on pp. 105-106) seems thorough and reasonable. It includes important steps related to 
training and oilotine that will increase the likelihood of success. It is nromisine that Battelle will be 
involved in pilot activities. Additionally, ODE provided a deadline by which local boards of 
education must pass ODE's guidelines. 


Weaknesses, issues, lack of clarity 


• ODE needs more information on technical assistance to districts. ODE may also need to do 
more than provide a clearinghouse of best practices. 


Technical Assistance Suggestions 


• ODE needs to demonstrate alignment of locally-designed systems and to determine what 

measures may be used for untested grades and subjects. These are major issues and getting them 
right will be essential for the success of the new evaluation system (p. 103). 


ii. Does the SEA's plan include sufficient involvement of teachers and principals in the development of these guidelines? 


3.A.i, Option A.ii Panel Response 

Not applicable because the SEA selected 3. A, Option B 
Tally of Peer Responses: X Yes, X No 


Response Component 


Peer Panel Response 


Rationale 




Strengths 




Weaknesses, issues, lack of clarity 




Technical Assistance Suggestions 





Hi. Note to Peers: Staff will review Hi. 
If the SEA selected Option B : 

If the SEA has developed and adopted all guidelines consistent with Principle 3: 



i. Are the guidelines the SEA has adopted likely to lead to the development of evaluation and support systems that increase the quality of 
instruction for students and improve student achievement? (See question 3A.ii to review the adopted guidelines for consistency with 
Principle 3.) 
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3.A.i, Option B.i Panel Response 

1X1 Not applicable because the SEA selected 3 A, Option A 
Tally of Peer Responses: X Yes, X No 



Response Component 


Peer Panel Response 


Rationale 




Strengths 




Weaknesses, issues, lack of clarity 




Technical Assistance Suggestions 





ii. Note to Peers: Staff will review ii. 



iii. Did the SEA have sufficient involvement of teachers and principals in the development of these guidelines? 



3.AA, Option B.iii Panel Response 

[ | Not applicable because the SEA selected 3 A, Option A 
Tally of Peer Responses: 3: Yes 3: No 



Response Component 


Peer Panel Response 


Rationale 


• There has been significant outreach to stakeholders but it is unclear how much stakeholder 
involvement shaped — or did not shape — the new systems. 


Strengths 


• ODE's outreach included: 73 principals participated in focus groups; 72 principals participated in 
train-the-trainers sessions (att. 7); 140 teachers and 120 evaluators participated in field testing of 
the model in 138 LEAs (att. 8); 18 meetings between the Governor's staff and teachers were held 
across the state. 
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Response Component 


Peer Panel Response 


Weaknesses, issues, lack of clarity 


• ODE is presently piloting its system in 138 schools; however, no rubrics were provided in 

conjunction with this information. There seems to be an inconsistency between the level of detail 
provided in the narrative and the components of the pilot system. ODE included only eight 
participants in its rubric design meetings. It is unclear to what extent the stakeholders' input, 
including educator input, has actually shaped the new teacher and principal evaluation systems, 
including: what ideas were generated or refined in these meetings that improved the proposed 
system? The contents and function of ODE's rubrics are unclear. It is unclear to what extent 
teachers of students with disabilities and English Learners will be included in this system. 
Additionally, it is unclear to what extent student data concerning students with disabilities and 
English Learners will be included in this system. 


Technical Assistance Suggestions 


• ODE needs to add examples of the content of stakeholder feedback (i.e., what did they like/ agree 
with in the new systems and what were their concerns?). ODE should explain the extent to 
which stakeholder concerns were addressed, or not, and why. ODE should develop a 
comprehensive timeline illustrating the plan for implementation by SY 2014-2015. ODE should 
ensure opportunities for ongoing educator involvement in design, implementation, and 
refinement of evaluation systems. 



ONLY FOR SEAs SELECTING OPTION B : If the SEA has adopted all guidelines for local teacher and principal evaluation and support 
systems by selecting Option B in section 3.A, review and respond to peer review question 3.A.ii below. 

3.A.ii Are the SEA's guidelines for teacher and principal evaluation and support systems consistent with Principle 3 — i.e., will they promote systems 
that: 



a. Will be used for continual improvement of instruction ? 

^ Are the SEA 's guidelines likely to result in support for all teachers, including teachers who are specialists working with students with disabilities and English 
Eearners and general classroom teachers with these students in their classrooms, that will enable them to improve their instructional practice? 
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3.A.ii.a Panel Response 

1X1 Not applicable because the SEA selected 3 A, Option A 
Tally of Peer Responses: X Yes, X No 



Response Component 


Peer Panel Response 


Rationale 




Strengths 




Weaknesses, issues, 
lack of clarity 




Technical Assistance 
Suggestions 





b. Meaningfully differentiate performance using at least three performance levels? 

^ Does the SEA incorporate student growth into its performance-level definitions with sufficient weighting to ensure that performance levels will differentiate 
among teachers and principals who have made significantly different contributions to student growth or closing achievement gaps? 



3.A.ii.b Panel Response 

1X1 Not applicable because the SEA selected 3 A, Option A 
Tally of Peer Responses: X Yes, X No 



Response Component 


Peer Panel Response 


Rationale 




Strengths 




Weaknesses, issues, lack 




of clarity 




Technical Assistance 




Suggestions 
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c. Use multiple valid measures in determining performance levels, including as a significant factor data on student growth for all students 
(including English Learners and students with disabilities), and other measures of professional practice (which may be gathered through 
multiple formats and sources, such as observations based on rigorous teacher performance standards, teacher portfolios, and student and 
parent surveys)? 

(i) Does the SEA have a process for ensuring that all measures that are included in determining performance levels are valid measures, 
meaning measures that are clearly related to increasing student academic achievement and school performance, and are implemented in 
a consistent and high-quality manner across schools within an LEA? 



3.A.ii.c(i) Panel Response 

1X1 Not applicable because the SEA selected 3 A, Option A 
Tally of Peer Responses: X Yes, X No 



Response Component 


Peer Panel Response 


Rationale 




Strengths 




Weaknesses, issues, lack 
of clarity 




Technical Assistance 
Suggestions 





(ii) For grades and subjects in which assessments are required under ESEA section 1111(b) (3), does the SEA define a statewide approach 
for measuring student growth on these assessments? 



3.A.ii.c(ii) Panel Response 

1X1 Not applicable because the SEA selected 3 A, Option A 
Tally of Veer Responses: X Yes, X No 



Response Component 


Peer Panel Response 


Rationale 




Strengths 




Weaknesses, issues, lack 




of clarity 




Technical Assistance 




Suggestions 
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(iii) For grades and subjects in which assessments are not required under ESEA section 1111(b)(3), does the SEA either specify the 
measures of student growth that LEAs must use or select from or plan to provide guidance to LEAs on what measures of student 
growth are appropriate, and establish a system for ensuring that LEAs will use valid measures? 



3.A.ii.c(iii) Panel Response 

1X1 Not applicable because the SEA selected 3. A., Option A 
Tally of Veer Responses: X Yes, X No 



Response Component 


Veer Panel Response 


Rationale 




Strengths 




Weaknesses, issues, lack 




of clarity 




Technical Assistance 




Suggestions 





d. Evaluate teachers and principals on a regular basis? 



3.A.ii.d Panel Response 

1X1 Not applicable because the SEA selected 3. A, Option A 
Tally of Peer Responses: X Yes, X No 



Response Component 


Peer Panel Response 


Rationale 




Strengths 




Weaknesses, issues, lack 
of clarity 




Technical Assistance 
Suggestions 
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e. Provide clear, timely, and useful feedback, including feedback that identifies needs and guides professional development? 

^ Will the SEA 's guidelines ensure that evaluations occur with a frequency sufficient to ensure that feedback is provided in a timely manner to inform effective 
practice? 

^ Are the SEA 's guidelines likely to result in differentiated professional development that meets the needs of teachers? 



3.A.ii.e Panel Response 

1X1 Not applicable because the SEA selected 3 A, Option A 
Tally of Peer Responses: X Yes, X No 



Response Component 


Peer Panel Response 


Rationale 




Strengths 




Weaknesses, issues, lack 




of clarity 




Technical Assistance 




Suggestions 





f. Will be used to inform personnel decisions? 



3.A.ii.f Panel Response 

1X1 Not applicable because the SEA selected 3 A, Option A 
Tally of Peer Responses: X Yes, X No 



Response Component 


Peer Panel Response 


Rationale 




Strengths 




Weaknesses, issues, lack 




of clarity 




Technical Assistance 




Suggestions 
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3. B Ensure LEAs Implement Teacher and Principal Evaluation and Support Systems 



3.B Is the SEA's process for ensuring that each LEA develops, adopts, pilots, and implements, with the involvement of teachers and principals, 

evaluation and support systems consistent with the SEA's adopted guidelines likely to lead to high-quality local teacher and principal evaluation 
and support systems? 

Does the SEA have a process for reviewing and approving an LEA 's teacher and principal evaluation and support systems to ensure that they are consistent with 
the SEA's guidelines and will result in the successful implementation of such systems? 

^ Does the SEA have a process for ensuring that an LEA develops, adopts, pilots, and implements its teacher and principal evaluation and support systems with the 
involvement of teachers and principals'? 

^ Did the SEA describe the process it will use to ensure that all measures used in an LEA 's evaluation and support systems are valid, meaning measures that are 
clearly related to increasing student academic achievement and school performance, and are implemented in a consistent and high-quality manner across schools within 
an LEA (i.e., process for ensuring inter-rater reliability)? 

^ Does the SEA have a process for ensuring that teachers working with special populations of students, such as students with disabilities and English Learners, are 
included in the LEA 's teacher and principal evaluation and support systems? 

y Is the SEA 's plan likely to be successful in ensuring that LEAs meet the timeline requirements by either (1) piloting evaluation and support systems no later than 
the 201 3— 2014 schoolyear and implementing evaluation and support systems consistent with the requirements described above no later than the 2014— 201 5 school 
year; or (2) implementing these systems no later than the 201 3— 2014 schoolyear? 

y Do timelines reflect a clear understanding of what steps will be necessary and reflect a logical sequencing and spacing of the key steps necessary to implement 
evaluation and support systems consistent with the required timelines? 

^ Is the SEA plan for providing adequate guidance and other technical assistance to LEAs in developing and implementing teacher and principal evaluation and 
support systems likely to lead to successful implementation? 

y Is the pilot broad enough to gain sufficient feedback from a variety of types of educators, schools, and classrooms to inform full implementation of the LEA's 
evaluation and support systems? 
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3.B Panel Response 

Tally of Peer Responses: 6: Yes 0: No 



Response Component 


Peer Panel Response 


Rationale 


• The new evaluation system will be the core of a comprehensive approach for improving teacher and 
principal quality and effectiveness. It will be used not only for hiring and placement but also for 
professional development, equitable distribution, differentiated roles and responsibilities, and performance- 
based compensation and tenure decisions. 


Strengths 


• Through its Race to the Top application, ODE developed a process for ensuring its LEAs develop, adopt, 
and pilot teacher and principal evaluation and support systems with the involvement of teachers and 
principals. All local boards of education are required to adopt policies with input of teacher and principals 
that comply with the state framework by July 1, 2012. Implementation is slated for SY 2013-2014 (p. 107). 

• ODE requires its LEAs to meet the timeline requirements by developing a process for LEAs to submit 
documentation of the implementation date prior to the July 1 deadline required by Ohio's state law. 
Presumably this means implementation will occur starting in SY 2013-2014 (p. 107). 

• ODE's timelines reflect awareness of what steps will be necessary to implement evaluation and support 
systems consistent with the required timelines. For example, the timelines include important steps such as 
conducting an external evaluation to get feedback through focus groups and surveys, looking at inter-rater 
reliability, and planning for use of data to support instructional and human capital decisions, etc. The 
process also ensures a demonstrated commitment to sufficient training and credentialing for evaluators, 
which is an essential step in the process (p. 108). 
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Response Component 


Peer Panel Response 


Weaknesses, issues, lack 
of clarity 


• ODE did not describe in sufficient detail the process used in field testing to ensure that all measures used in 
an LEA's evaluation and support systems are valid. It is not clear that there is a process for ensuring inter- 
rater reliability, nor is it clear who even did the field testing (i.e., were there external validators?). 

• Union leaders' signatures do not necessarily indicate support from rank and file teachers. ODE's request 
also lacks sufficient detail as to when and how teachers will be included as the process of developing and 
implementing the evaluation systems continues (p. 107). 

• ODE does not demonstrate that it has a process for ensuring that teachers working with special populations 
of students, such as students with disabilities and English Learners, are included in the LEA's teacher and 
principal evaluation and support systems. 


Technical Assistance 
Suggestions 


• ODE should add further detail on the process that will be used in field testing to ensure that all measures 
used in an LEA's evaluation and support systems are valid, especially to ensure inter-rater reliability. 

• ODE should demonstrate that it has an adequate process for ensuring that teachers working with special 
populations of students, such as students with disabilities and English Learners, are included in the LEA's 
teacher and principal evaluation and support systems. 



Principle 3 Overall Review 



If the SEA indicated that it has not developed and adopted all guidelines for local teacher and principal evaluation and support systems consistent with 
Principle 3 by selecting Option A in section 3.A, is the SEA's plan for the SEA's and LEAs' development and implementation of teacher and principal 
evaluation and support systems comprehensive, coherent, and likely to increase the quality of instruction for students and improve student 
achievement? If not, what aspects are not addressed or need to be improved upon? 

If the SEA indicated that is has adopted guidelines for local teacher and principal evaluation and support systems consistent with Principle 3 by 
selecting Option B in section 3.A, are the SEA's guidelines and the SEA's process for ensuring, as applicable, LEA development, adoption, piloting, and 
implementation of evaluation and support systems comprehensive, coherent, and likely to increase the quality of instruction for students and improve 
student achievement? If not, what aspects are not addressed or need to be improved upon? 
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Principle 3 Overall Review Panel Response 

Tally of Peer Responses: 5: Yes h No 



Response Component 


Peer Panel Response 


Rationale 


• The new evaluation system can be a critical component of ODE's efforts to increase student achievement through 
improving teacher and principal quality and effectiveness. The reforms underway in this area are likely to yield 
improvements in decisions related to hiring, placement, professional development, equitable distribution of teachers, 
differentiated roles, performance-based compensation, and tenure. 


Strengths 


• ODE's guidelines are formally being adopted in July 2012 and the implementation timeline (as described on pp. 105- 
1 06) seems thorough and reasonable. It includes important steps related to training and piloting that will increase the 
likelihood of success. 


Weaknesses, issues, lack 
of clarity 


• ODE needs more information on the technical assistance that will be provided to districts. ODE may also need to 
do more than provide a clearinghouse of best practices. Details were not provided as to specific development of new 
growth measures and support for fidelity of implementation. No plans are evident for continued solicitation of 
stakeholder input. 


Technical Assistance 
Suggestions 


• ODE needs to demonstrate a process for ensuring alignment of locally-designed systems with its guidelines. 

• ODE should, in conjunction with educators, determine what measures may be used for untested grades and subjects. 

• ODE should describe a process for continued stakeholder input. 
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Overall Request Evaluation 



Did the SEA provide a comprehensive and coherent approach for implementing the waivers and principles in its request for the flexibility? Overall, is 
implementation of the SEA's approach likely to increase the quality of instruction for students and improve student achievement? If not, what aspects 
are not addressed or need to be improved upon? 



Overall Request Evaluation Panel Response 
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Peer Panel Response 


Rationale 


• 


ODE has developed a comprehensive plan for transition to college- and career-ready standards and implementation 
by 2013-14; a differentiated accountability system that takes into consideration supports and interventions for all 
schools; and is progressing towards development and implementation of teacher and principal evaluation systems. 
Concerns exist surrounding the limited representation of students with disabilities, English Learners, and low- 
performing students throughout the plan in ways that will fully include and support their improved achievement. 




• 


ODE uses a theory of action to organize and communicate the system to stakeholders. 




• 


This flexibility request addresses the requirements, except for one attachment that is still expected. 


Strengths 


• 


While stakeholder outreach was evident and ranged from legislators to classroom teachers and parents, it is not clear 
to what extent stakeholder feedback was used to shape the request. 




• 


The various activities were generally reasonable and should move the state towards accomplishing its designated goals 
and tasks. 




• 


ODE should demonstrate increased commitment to high-needs populations, including low-income students, students 
with disabilities, and English Learners. Extended standards are applicable to a group of students with disabilities that 
is too broad. 


Weaknesses, issues, 
lack of clarity 


• 


ODE did not demonstrate appropriate alignment between the interventions and desired outcomes in the 
differentiated recognition, accountability and supports. 




• 


The timelines were often unclear or overly ambitious and should be carefully considered in order to move ODE 
toward accomplishing the designated goals and tasks. 
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Response Component 


Peer Panel Response 


Technical Assistance 
Suggestions 


• ODE should articulate a unified, coherent, and inclusive plan, tied to the State's theory of action, that will facilitate 
Ohio's ability to achieve its goals. 
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